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VIDEO DATA COMPRESSION APPARATUS AND METHOD OF SAME 



BACKGROUND OF THE INVENTION 

1. Field of the Invention 

The present invention relates to a video data 
compression apparatus for compressing and coding 
noncompressed video data and a method of same. 

2. Description of the Related Art 

When compressing and coding noncompressed 
digital video data in units of GOPs (groups of pictures) 
comprised of an I picture (intra-coded picture), a B 
picture (bi-directionally predictive coded picture), and d 
P picture (predictive coded picture) by a method such as 
the MPEG (Moving Picture Experts Group) and recording the 
same on a recording medium such as a magneto -optical disc 
(MO disc) or transmitting the same via communication line, 
it is necessary to control the amount of data (amount of 
bits) of the compressed video data after the compression 
and coding to be not more than the recording capacity of 
the recording medium or not more than the transmission 
capacity of the communication line while holding the 
quality of the image after expansion and decoding high. 

For this purpose, first, there is adopted a 
method of preliminarily compressing and coding the 



noncompressed video data to estimate the amount of data 
after the compression and coding (first pass), then 
adjusting the compression rate based on the estimated 
amount of data and performing the compression and coding 
so that the amount of data after the compression and 
coding becomes not more than the recording capacity of the 
recording medium by (second pass) (hereinafter, such a 
compression and coding method will be also referred to as 
"two pass encoding" ) . 

However, if performing the compression and 
coding by the two pass encoding, it is necessary to apply 
similar compression and coding processing two times with 
respect to the same noncompressed video data, so a long 
time is taken. Further, since the final compressed video 
data cannot be calculated by one compression and coding 
processing, the shot video data cannot be compressed, 
coded, recorded, or transmitted in real time as it is. 
SUMMARY OF THE INVENTION 

An object of the present invention is to provide a 
video data compression apparatus with which the audio 
and/or video data can be compressed and coded to a 
predetermined amount of data or less not by the two pass 
encoding and a method of the same. 

Further, another object of the present invention is to 
provide a video data compression apparatus with which the 
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video data can be compressed and coded in almost real 
time and in addition with which a high quality image can 
be obtained after the expansion and decoding and a method 
of the same. 

Further, still another object of the present 
invention is to provide a. video data compression 
apparatus with which the compression and coding 
processing can be carried out by adjusting the 
compression rate by estimating the amount of data after the 
compression and coding not by the two pass encoding and a 
method of the same. 

According to a first aspect of the present 
invention, there is provided a video data compression 
apparatus having an indicator data calculating means for 
calculating indicator data indicating a complexity of 
video data for every picture from noncompressed video 
data; a target value calculating means for calculating a 
target value of an amount of data after compression of 
the video data for every picture based on the calculated 
indicator data; and a compressing means for compressing the 
noncompressed video data so that the amount of data after 
compression becomes the calculated target value. 

According to a second aspect of the present 
invention, there is provided a data compression method 
comprising the steps of calculating indicator data 
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Indicating a complexity of video data for every picture 
from noncompressed video data; calculating a target value 
of an amount of data after compression of the video data 
for every picture based on the calculated indicator data; 
5 and compressing the video data by a predetermined 
compression method so that the amount of data after 
compression becomes the calculated target value. 
According to a third aspect of the present 

111 

f: invention, there is provide a video data compression 

m 

* ? " 10 apparatus having an indicator data calculating means for 

calculating indicator data indicating a complexity of 

n § 

12; video data for every picture; a difficulty data 

fT calculating means for performing a predetermined 

computation processing for multiplying a coefficient with 

15 the calculated indicator data to calculate difficulty 
data corresponding to the amount of data after 
compression of the video data; a target value calculating 
means for calculating a target value of the amount of 
data after compression of the video data for every 

20 picture based on the calculated difficulty data; a 

compressing means for compressing each of the pictures of 
the video data by a predetermined compression method so 
that the amount of data after compression becomes the 
calculated target value so as to generate compressed 

25 video data; and a coefficient updating means for updating 
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the coefficient based on the amount of data of the 
generated compressed video data* 

According to a fourth aspect of the present 
invention, there is provided a video data compression 
5 method comprising the steps of calculating indicator data 
indicating a complexity of video data for every picture; 
performing predetermined computation processing for 

1 

|P multiplying a coefficient with the calculated indicator 

data to calculate difficulty data corresponding to the 
10 amount of data after compression; calculating a target 
H value of the amount of data after compression of the 

iy 

irl ! 

IS noncompressed video data for every picture based on the 

H calculated difficulty data; compressing each of the 

pictures of the video data by the compression method so 
15 that the amount of data after compression becomes the 
calculated target value so as to generate compressed 
video data; and updating the coefficient based on the 
amount of data of the generated compressed video data. 
According to a fifth aspect of the present 
20 invention, there is provided a video data compression 

apparatus for compressing a continuous plurality of video 
data to compressed video data of a picture type sequence 
containing a plurality of types of pictures (I picture, P 
picture, and B picture) in a predetermined order, having a 
25 rearranging means for rearranging pictures of the 
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noncompressed video data to an order adapted to the 
compression method so that each head picture of the video 
data becomes an I picture or a P picture; an indicator 
data calculating means for calculating indicator data 
5 indicating a complexity of the rearranged noncompressed 
video data for every picture; a border detecting means 
for detecting a scene change of a continuous plurality of 
the noncompressed video data; a changing means for 
changing the picture type sequence so that pictures of 

10 any of the noncompressed video data are compressed 
without reference to the pictures of the other 
noncompressed video data for every border of a detected 
scene change; a target value calculating means for 
calculating a target value of the amount of data after 

15 compression of the video data based on the calculated 
indicator data and the picture type sequence after 
change; and a compressing means for compressing the video 
data to compressed video data of the picture type 
sequence after change so that the amount of data after 

20 compression becomes substantially the calculated target 
value , 

According to a sixth aspect of the present 
invention, there is provided a video data compression 
method for compressing a continuous plurality of video 
25 data to compressed video data of a picture type sequence 



containing a plurality of types of pictures (I picture , P 
picture , and B picture) in a predetermined order, 
comprising the steps of rearranging pictures of the 
noncompressed video data to an order adapted to the 
compression method so that each head picture of the video 
data becomes an I picture or a P picture; calculating 
indicator data indicating a complexity of the rearranged 
noncompressed video data for every picture; detecting a 
scene change of the continuous plurality of the 
noncompressed video data; changing the picture type 
sequence so that a picture of any of the noncompressed 
video data is compressed without reference to a. pictures 
of other noncompressed video data for every border of a 
detected scene change; calculating a target value of the 
amount of data after compression of the video data for 
every predetermined compression unit based on the 
calculated indicator data and the picture type sequence 
after change; and compressing the video data to the 
compressed video data of the picture type sequence after 
change so that the amount of data after compression 
becomes substantially the calculated target value. 
Note that flatness is defined as an indicator 
representing a spatial flatness of van image, represents 
the complexity of the image, and has a correlation with the 
difficulty (degree of difficulty) of the pattern of the 
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Image and the amount of data after compression* 

Further, an Intra AC ±s defined as the sura of the 
dispersion values of the average value of the video data 
in the pictures and the video data for every DCT block of 
5 the DCT processing unit in the MPEG system, represents 
the complexity of the image, and has a correlation with 
the difficulty (degree of difficulty) of the pattern of 
the image and the amount of data after compression 
similar to the flatness. 

10 Further, an ME residual is defined as the sum of 

absolute values or the sum of square values of the 
difference of the video data between the input picture 
and a reference picture after motion compensation 
processing by the motion vector at the compression and 

15 coding, represents the speed of the motion of the image 
and the complexity of the pattern, and has a correlation 
with the difficulty (degree of difficulty) of the pattern 
of the image and the amount of data after compression 
similar to the flatness, 

20 Further, the activity is defined as the activity 

used for determining a quantization value of quantization 
processing with respect to a macroblock in TM5 [test 
model 5; ISO/IEC JTC/SC29 (1993)] famous as a compression 
algorithm of the MPEG system, 

25 BRIEF DESCRIPTION OF THE DRAWINGS 
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These and other objects and features of the present 
invention will become clearer from the following 
description of the preferred embodiments given with 
reference to the attached drawings, in which: 
5 Fig- 1 is a view of the configuration of a video 

data compression apparatus according to the present 
invention ; 

Fig. 2 is a view of the configuration of an encoder 
of a simplified two pass processing unit shown in Fig, 1; 
10 Fig. 3 is a view of the configuration of the encoder 

shown in Fig. 1; 

Figs. 4A to 4C are views of an operation of the 
simplified two pass encoding of the video data 
compression apparatus according to a first embodiment of 
15 the present invention 

Fig. 5 is a schematic view of the configuration of 
the video data compression apparatus according to a 
second embodiment of the present invention; 

Fig. 6 is a detailed view of the configuration of a 
20 compression and coding unit of the video data compression 
apparatus shown in Fig. 5; 

Fig. 7 is a view of the correlation between an ME 
-residual and a real difficulty data when producing a P 
picture by the video data compression apparatus shown in 
25 Fig. 1 and Fig. 5; 
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Fig. 8 ±s a view of the correlation between the ME 
residual and the real difficulty data D J when producing a 
B picture by the video data compression apparatus shown 
in Fig. 1 and Fig, 5; 
5 Fig, 9 is a view of a method of calculation of 

flatness; 

Fig. 10 is a view of the correlation between the 
flatness and the real difficulty data D J when producing 
an I picture by the video data compression apparatus shown 

10 in Fig. 1 and Fig. 5; 

Fig. 11 is a view of the correlation between the 
flatness and the real difficulty data when producing 
an I picture by the video data compression apparatus shown 
in Fig. 1 and Fig. 5; 

15 Fig. 12 is a view of a compression and coding 

operation of the video data compression apparatus (Fig. 
6) according to a third embodiment of the present 
invention ; 

Fig. 13 is a view of the content of the processing 
20 of the host computer (Fig. 6) of the video data 

compression apparatus 2 according to the third embodiment 
of the present invention; 

Fig. 14 is a view of the compression and coding 
operation of the video data compression apparatus 
25 according to a fourth embodiment of the present invention 
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in an order of coding; 

Figs. 15A to 15C are views of a processing for 
rearrangement of the order of the pictures of the edited 
video data by an encoder control unit (Fig. 1) and 
5 processing for changing the picture type by the host 
computer; and 

Fig. 16 is a flowchart of the content of the 
processing of the host computer of the video data 
compression apparatus according to a fourth embodiment of 
10 the present invention 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

First Embodiment 

Below, a first embodiment of the present invention 
will be explained. 

15 If a plurality of video data having a high coding 

difficulty such as a pattern having a large number of 
high frequency components or a pattern having a large 
amount of motion are compressed and coded by a 
compression and coding system of video data such as the 

20 MPEG system, generally distortion accompanying the 
compression is apt to occur. For this reason, it is 
necessary to compress and code video data having a high 
difficulty with a low compression rate and allocate a 
larger target amount of data with respect to the 

25 compressed video data obtained by compressing and coding 
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the data having a high difficulty in comparison with 
compressed video data of video data of a pattern having a 
low difficulty. 

In order to adaptively allocate the target amount 
5 of data with respect to the difficulty of the video data 

in this way, the two pass encoding system indicated as the 
related art ±s effective* However, the two pass encoding 
system is not suited to real time compression and coding. 
The simplified two pass encoding system indicated as 
10 the first embodiment was devised so as to solve the 

disadvantage of such a two pass encoding system. It can 
calculate the difficulty of the noncompressed video data 
from the difficulty data of the compressed video data 
obtained by preliminarily compressing and coding the 
15 noncompressed video data and adaptively control the 

compression rate of the noncompressed video data delayed 
by exactly a predetermined time by an FIFO memory or the 
like based on the difficulty calculated by the 
preliminary compression and coding. 
20 Figure 1 is a view of the configuration of a video 

data compression apparatus 1 according to the present 
invention. 

As shown in Fig. 1, the video data compression 
apparatus 1 is constituted by a compression and coding 
25 unit 10 and a host computer 20. The compression and coding 
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unit 10 ±s constituted by an encoder control unit 12, a 
motion detector 14, a simplified two pass processing unit 
16, and a second encoder 18- The simplified two pass 
processing unit 16 is constituted by an FIFO memory 160 
and a first encoder 162* 

The video data compression apparatus 1 realizes the 
above simplified two pass encoding with respect to the 
noncompressed video data VIN input from an external 
apparatus (not illustrated) such as an editing apparatus 
and a video tape recorder apparatus by these constituent 
parts „ 

In the video data compression apparatus 1, the host 
computer 20 controls the operation of the constituent 
parts of the video data compression apparatus 1, Further, 
the host computer 20 receives the amount of data of the 
compressed video data generated by preliminarily 
compressing and coding the noncompressed video data VIN 
by the encoder 162 of the simplified two pass processing 
unit 16 . the value of the direct current component (DC 
component) of the video data after the DCT processing, 
and the power value of the alternating current component 
(AC component) via a control signal C16 and calculates 
the difficulty of the pattern of the compressed video 
data based on these received values. Further, the host 
computer 20 allocates the target amount of data Tj of the 




compressed video data generated by the encoder 18 via the 
control signal C18 for every picture based on the 
calculated difficulty, sets the same in the quantization 
circuit 166 (Fig. 3) of the encoder 18, and adapt ively 
controls the compression rate of the encoder 18 in units 
of pictures . 

The encoder control unit 12 notifies the existence 
of a picture of the noncompressed video data VIN to the 
host computer 20 and further performs the preprocessing 
for the compression and coding for every picture of the 
noncompressed video data VIN. That is, the encoder 
control unit 12 rearranges the input noncompressed video 
data in the order of coding, performs picture field 
transformation, performs 3:2 pull down processing 
(processing for transf orming the video data of 30 
frames/sec to video data of 24 frames/sec) , etc. when the 
noncompressed video data VIN is the video data of a movie 
and outputs the same as the video data S12 to the FIFO 
memory 160 and the encoder 162 of the simplified two pass 
processing unit 16. 

The motion detector 14 performs the detection of the 
motion vector of the noncompressed video data and outputs 
the same to the encoder control unit 12 and the encoders 
162 and 18. 

In the simplified two pass processing unit 16, the 
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FIFO memory 160 delays the video data S12 input from the 
encoder control unit 12 by exactly a time by which for 
example L (L is an integer) number of pictures of the 
noncompressed video data VIN is input and outputs the 
same as the delayed video data S16 to the encoder 18. 

Figure 2 is a view of the configuration of the 
encoder 162 of the simplified two pass processing unit 16 
shown in Fig. 1. 

The encoder 162 is a video data use compression and 
coding unit constituted by, for example, as shown in Fig. 
2, an adder circuit 164, a DCT circuit 166, a 
quantization circuit (Q) 168, a variable length coding 
circuit (VLC) 170, an inverse quantization circuit (IQ) 
172, an inverse DCT (IDCT) circuit 174, an adder circuit 
176, and a motion compensation circuit 178, compresses 
and codes the input video data S12 by the MPEG system or 
the like, and outputs the amount of data etc. for every 
picture of the compressed video data to the host computer 
20. 

The adder circuit 164 subtracts the output data of 
the adder circuit 176 from the video data S12 and outputs 
the result to the DCT circuit 166. 

The DCT circuit 166 performs discrete cosine 
transformation (DCT) processing on the video data input 
from the adder circuit 164 in units of DCT blocks 
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consisting of for example 8 pixels x 8 pixels, transforms 
the same from data of the time domain to data of the 
frequency domain, and outputs the result to the 
quantization circuit 168, Further, the DCT circuit 166 
outputs the value of the DC component of the video data 
after DCT and the power value of the AC component to the 
host computer 20 . 

The quantization circuit 168 quantizes the data of 
the frequency domain input from the DCT circuit 166 by 
the fixed quantization value Q and outputs the result as 
the quantized data to the variable length coding circuit 
170 and the inverse quantization circuit 172. 

The variable length coding circuit 170 performs 
variable length coding on the quantized data input from 
the quantization circuit 168 and outputs the amount of data 
of the compressed video data obtained as a result of the 
variable length coding via the control signal C16 to the 
host computer 20. 

The inverse quantization circuit 172 performs 
inverse quantization on the quantized data input from the 
variable length coding circuit 168 and outputs the same 
as the inverse quantized data to the inverse DCT circuit 
174. 

The inverse DCT circuit 174 performs inverse DCT 
processing on the inverse quantized data input from the 
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Inverse quantization circuit 172 and outputs the result 
to the adder circuit 176. 

The adder circuit 176 adds the output data of the 
motion compensation circuit 178 and the output data of 
5 the inverse DCT circuit 174 and outputs the result to the 
adder circuit 164 and the motion compensation circuit 178. 

The motion compensation circuit 178 performs the 
motion compensation processing based on the motion vector 
input from the motion detector 14 on the output data of 

10 the adder circuit 176 and outputs the result to the adder 
circuit 176. 

Figure 3 is a view of the configuration of the 
encoder 18 shown in Fig. 1. 

As shown in Fig. 3, the encoder 18 is configured as 

15 the encoder 162 shown in Fig. 2 plus the quantization 
control circuit 180. The encoder 18 applies the motion 
compensation processing, DCT processing, quantization 
processing, and variable length coding processing with 
respect to the delayed video data S16 delayed by the 

20 amount of L pictures by the FIFO memory 160 by these 

constituent parts based on the target amount of data T^ 
set from the host computer 20 to generate the compressed 
video data VOUT of the MPEG system or the like and 
outputs the result to an external apparatus (not 

25 illustrated) . 
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In the encoder 18, the quantization control circuit 
180 successively monitors the amount of data of the 
compressed video data VOUT output by the variable length 
quantization circuit 170 and successively adjusts the 
quantization value to be set in the quantization 
circuit 168 so that the amount of data of the compressed 
video data finally generated from the j-th picture of the 
delayed video data S16 approaches the target amount of 
data Tj set from the host computer 20. 

Further, the variable length quantization circuit 
170 outputs the compressed video data VOUT to the outside 
and in addition outputs the actual amount of data of 
the compressed video data VOUT obtained by compressing 
and coding the delayed video data S16 via the control 
signal C18 to the host computer 20. 

Below, an explanation will be made of the simplified 
two pass encoding operation of the video data compression 
apparatus 1 in the first embodiment. 

Figures 4A to 4C are views of the operation of the 
simplified two pass encoding of the video data 
compression apparatus 1 in the first embodiment. 

The encoder control unit 12 performs the 
preprocessing of rearrangement of pictures in the order 
of coding etc. with respect to the noncompressed video data 
VIN input to the video data compression apparatus 1 by 
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the encoder control unit 12 and outputs the same as the 
video data S12 to the FIFO memory 160 and the encoder 162 
as shown in Fig. 4A. 

Note that, due to the rearrangement of the order of 
pictures by the encoder control unit 12, the order of 
coding of the pictures shown in Figs. 4A to 4C etc. and 
the order indicated after expansion and decoding are 
different. 

The FIFO memory 160 delays the pictures of the input 
video data S12 by exactly the amount of Ii pictures and 
outputs the same to the encoder 18. 

The encoder 162 preliminarily successively 
compresses and codes the pictures of the input video data 
S12 and outputs the amount of data of the compressed and 
coded data obtained by compressing and coding the j (j is 
an integer) -th picture, the value of the DC component of 
the video data after the DCT processing, and the power 
value of the AC component to the host computer 20. 

For example, since the delayed video data S16 input 
to the encoder 18 is delayed by exactly the amount of L 
pictures by the FIFO memory 160 # as shown in Fig. 4B, 
when the encoder 18 compresses and codes the j (j is an 
integer) -th picture (picture a of Fig. 4B) of the delayed 
video data S16, the encoder 162 compresses and codes the 
(j+L)-th picture (picture b of Fig. 4B) advanced from the 
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3 -til picture of the video data S12 by the amount of L 
pictures* Accordingly, when the encoder 18 starts the 
compression and coding of the j-th picture of the delayed 
video data S16, the encoder 162 has completed the 
5 compression and coding of the j-th to (j-fL-l)-th pictures 
(range c of Fig. 4B) of the video data S12, and the real 
difficulty data , I> j+2 , , d 3+ l-i after compression 

and coding of these pictures have been already calculated 
by the host computer 20. 

10 The host computer 20 calculates the target amount of 

data to be allocated to the compressed video data 
obtained by compressing and coding the j-th picture of 
the delayed video data S16 by the encoder 18 by the 
following Equation 1 and sets the calculated target 

15 amount of data in the quantization control circuit 180. 



Note, in Equation 1, D j is the real difficulty data 
of the j-th picture of the video data S12 # R f ^ is an 
average of the target amount of data which can be allocated 
to the j-th to the (j+L-l)-th pictures of the video data 
2 0 S12 and S16, and an initial value (R 1 x ) of R f 3 is the 

target amount of data which can be allocated to pictures 
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of the compressed video data on the average, represented 
by following Equation 2, and updated as shown in Equation 
3 whenever the encoder 18 generates one picture's worth 
of the compressed video data, 

R\ = (Bit_rate x L ) /Picture__rate (2) 

Note that, the numerical value bit rate (Bit rate) 
in Equation 3 indicates the amount of data (amount of 
bits) per* second determined based on the transmission 
capacity of the communication line and the recording 
capacity of the recording medium, "Picture rate" 
indicates the number of pictures per second contained in 
the video data (30 pictures/sec (NTSC) # 25 pictures/sec 
(PAL)), and the numerical value F j+L indicates the average 
amount of data per picture determined in accordance with 
the picture type. 

The DCT circuit 166 of the encoder 18 performs DCT 
processing on the j-th picture of the input delayed video 
data S16 and outputs the result to the quantization 
circuit 168. 

The quantization circuit 168 quantizes the data of 
the frequency domain of the j-th picture input from the 
DCT circuit 166 by the quantization value adjusted 
based on the target amount of data by the quantization 
control circuit 180 and outputs the result as the 
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quantized data to the variable length, coding circuit 170. 

The variable length coding circuit 170 performs 
variable length coding on the quantized data of the j-th 
picture input from the quantization circuit 168 so as to 
5 generate the compressed video data VOUT of an amount of 
data near the target amount of data T j and outputs the 
same. 

Similarly, as shown in Fig. 4B # when the encoder 18 
compresses and codes the (j+l)-th picture (picture a 1 of 

10 Fig. 4C) of the delayed video data S16, the encoder 162 
completes the compression and coding of the (j+l)-th to 
(j+L)-th pictures (range c 1 of Fig. 4C) of the video data 
S12. The real difficulty data D j+1 , D J+2 , D j+3 , "\ D J+L . t of 
these pictures have been already calculated by the host 

15 computer 20. 

The host computer 20 calculates the target amount of 
data T j+1 to be allocated to the compressed video data 
obtained by compressing and coding the (j+l)-th picture 
of the delayed video data S16 by the encoder 18 by 

20 Equation 1 and sets the same in the quantization control 
circuit 180 of the encoder 18. 

The encoder 18 compresses and codes the (j+l)-th 
picture based on the target amount of data Tj set in the 
quantization control circuit 180 from the host computer 

25 20 so as to generate the compressed video data VOUT of an 
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amount of data near the target amount of data T j+1 and 
outputs the same. 

Further, below, similarly, the video data 
compression apparatus 1 successively compresses and codes 
5 the k-th picture of the delayed video data S16 by 

changing the quantization value Q k (k = 3 + 2, j+3, * ") for 
every picture and outputs the result as the compressed 
video data VOUT. 

As explained above, according to the video data 

10 compression apparatus 1 shown in the first embodiment, 

the difficulty of the pattern of the noncompressed video 
data VIN can be calculated in a short time and the 
noncompressed video data VIN can be adaptively compressed 
and coded with a compression rate in accordance with the 

15 calculated difficulty- That is, according to the video 
data compression apparatus 1 shown in the first 
embodiment, the noncompressed video data VIN can be 
adaptively compressed and coded in almost real time based 
on the difficulty of the pattern of the noncompressed 

20 video data VIN unlike the two pass encoding system, and 

thus the apparatus can be applied to applications such as 
live broadcasts in which a real time property is required- 

Note that, in addition to that shown in the first 
embodiment, the data video data compression apparatus 1 

25 according to the present invention can adopt various other 
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configurations, for example, the amount of data of tlie 
compressed video data compressed and coded by tlie encoder 
162 may be used as it is as the difficulty data so as to 
simplify the processing of the host computer 20. 
5 Second Embodiment 

Below, a second embodiment of the present invention 
P| will be explained, 

~ The simplified two pass encoding system shown in the 

first embodiment is an excellent system capable of 
fjfj 10 performing compression and coding by giving only a delay of 

|) about one GOP (for example 0 . 5 second) to the input 

f|1 noncompressed video data and generating compressed video 

I j data of suitable amounts of data* 

^ In this system, however, two encoders are required. 

15 In general, an encoder for compressing and coding video 
data requires large hardware and is very expensive even 
if formed as an integrated circuit. In addition, it is 
large in size. Accordingly, the necessity of two encoders 
in this system prevents a reduction of the cost of the 

20 apparatus, a reduction of size, and saving of electric 
power. Further, it is desirable that the time delay 
required for the compression and coding be as short as 
possible, but the processing for calculation of the real 
difficulty data and the predictive difficulty data 1 

25 and the preliminary compression and coding processing per 
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se require a few pictures* worth, of processing time, 
therefore these processings per se become a factor 
preventing the shortening of the time delay. 

The second embodiment was made so as to solve such a 
disadvantage and provides a video data compression method 
capable of generating compressed video data of suitable 
amounts of data equivalent to the simplified two pass 
encoding system by only using one encoder and in addition 
requiring a shorter time delay for the processing. 

Figure 5 is a schematic view of the configuration of 
a video data compression apparatus 2 according to a 
second embodiment of the present invention . 

Figure 6 is a detailed view of the configuration of 
a compression and coding unit 24 of the video data 
compression apparatus 2 shown in Fig, 5. 

Note that, in Fig. 5 and Fig. 6, among the 
constituent parts of the video data compression apparatus 
2, the parts the same as the constituent parts of the 
video data compression apparatus 1 explained in the first 
embodiment (Fig. 1 and Fig. 2) are indicated by same 
symbols or numerals. 

As shown in Fig. 5, the video data compression 
apparatus 2 adopts a configuration in which the 
compression and coding unit 10 of the video data 
compression apparatus 1 (Fig. 1 and Fig. 2) is replaced 
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by a compression and coding unit 24 obtained by excluding 
the encoder 162 from the compression and coding unit 10, 
replacing the encoder control unit 12 by the encoder 
control unit 22 , and adding a buffer memory (buffer) 182. 
5 As shown in Fig. 6, the compression and coding unit 

24 is constituted by an image rearranging circuit 220 , a 
scanning transforming and macroblock forming circuit 222 r 
and a statistical amount (parameter) calculation circuit 
224. The rest of the constituent parts of the compression 

10 and coding unit 24 adopt the same configurations as those 
of the compression and coding unit 10. 

The encoder control unit 22 notifies the existence 
of a picture of the noncompressed video data VIN to the 
host computer 20 similar to the encoder control unit 12 

15 and further performs the preprocessing for compression 
and coding for every picture of the noncompressed video 
data VIN. 

In the encoder control unit 22 , the image 
rearranging circuit 220 rearranges the input 
20 noncompressed video data in the order of coding. 

The scanning transforming and macroblock forming 
circuit 222 performs the picture field transformation and 
performs the 3:2 pull down processing etc. where the 
source of the noncompressed video data VIN is the video 
25 data of a movie. 
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The statistical amount calculation circuit 224 
calculates a statistical amount such as the flatness and 
intra AC from the pictures processed by the image 
rearranging circuit 220 and the scanning transforming and 
5 macroblock forming circuit 222 and compressed and coded 
to I pictures. 

The video data compression apparatus 2 uses a 
statistical amount (flatness, intra AC) of the 
noncompressed video data and the predictive error amount 

10 (ME residual) of motion prediction in place of the 

difficulty of the pattern of the noncompressed video data 
VIN by these constituent parts, adaptively calculates the 
target amount of data similar to the video data 
compression apparatus 1 (Fig. 1 and Fig. 2), and performs 

15 high precision feed forward control to compress and code 
the noncompressed video data VIN to compressed video data 
of suitable amounts of data. 

Note that, in the video data compression apparatus 
2, the target amount of data T., is determined by the 

20 motion detector 14 and the statistical amount calculation 
circuit 224 of the encoder control unit 22 based on the 
indicator data (parameter) detected in advance. 
Therefore, below, the compression and coding system in the 
video data compression apparatus 2 will be referred to as 

25 a feed forward rate control (FFRC) system. 



- 28 - 



Note that, the ME residual is defined as the sum of 
the absolute values or the sum of the square values of 
the difference of the video data between pictures to be 
compressed and a reference picture, is calculated from 
the pictures which become the P pictures and B pictures 
after compression by the motion detector 14 r represents 
the speed of the motion of the image and the complexity 
of the pattern, and has a correlation with the difficulty 
and the amount of data after compression similar to the 
flatness . 

The I picture is compressed and coded without 
reference to other pictures, so the ME residual cannot be 
found* Therefore, the flatness and intra AC are used as the 
parameters in place of the ME residual ♦ 

Further, flatness is a parameter newly defined as an 
indicator representing the spatial flatness of the image 
so as to realize the video data compression apparatus 2, 
indicates the complexity of the video, and has a 
correlation with the difficulty (degree of difficulty) of 
the pattern of the image and the amount of data after the 
compre s s ion • 

Further, the intra AC is a parameter newly defined 
as the sum of the dispersion values of the video data for 
every DCT block of the DCT processing unit in the MPEG 
system so as to realize the video data compression 
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apparatus 2, indicates the complexity of the image, and. 
has a correlation with the difficulty of the pattern of 
the image and the amount of data after the compression 
similar to the flatness. 

Below, an explanation will be made of the ME 
residual, flatness, and intra AC. 

In the simplified two pass encoding system explained 
41 in the first embodiment, the real difficulty data 

:Lf: jj 

indicated the difficulty of the pattern of the image and 

' s ?l 

10 the target amount of data was calculated based on the 
13 real difficulty data . 

In order to make the amount of data of the 
compressed video data generated by the encoder 18 
approach the value indicated by the target amount of data 

w the quantization value is controlled in the 
quantization circuit 168 (Fig. 2 and Fig. 6). 
Accordingly, if a parameter obtained without compressing 
and coding the video data and adequately indicating the 
complexity (difficulty) of the video data similar to the 
20 real difficulty data can be obtained before the 

quantization processing in the quantization circuit 168 
of the encoder 18, it would be possible to achieve the 
objectives of eliminating the encoder 162 (Fig. 1) and 
shortening the processing delay time. The ME residual, 
25 the flatness, and the intra AC have a strong correlation 
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with the real difficulty data and therefore are 
adequate for achieving these objectives. 

Relationship Between ME Residual and Real Difficulty 



referring to other pictures to generate the P pictures 
and B pictures, the motion detector 14 searches for a 
macroblock giving the smallest sum of the absolute values 
or sum of the square values of the difference between a 

10 particular macroblock of the picture (input picture) to 
be compressed and the picture referred to (reference 
picture) so as to find the motion vector* The ME residual 
is defined as the value obtained by adding up the sums of 
absolute values or sums of the squares of the differences 

15 of each of macroblocks giving the smallest sums for all 

of the pictures when finding the motion vector in this way. 

Figure 7 is a view of the correlation between the ME 
residual and the real difficulty data when generating 
a P picture by the video data compression apparatuses 1 

20 and 2. 

Figure 8 is a view of the correlation between the ME 
residual and the real difficulty data D 3 when generating 
a B picture by the video data compression apparatuses 1 
and 2. 

25 Note that, in Fig. 7 and Fig. 8, as the real 
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When performing compression and coding processing by 




difficulty data D^, use is made of the amount of data of 
the compressed video data obtained by compression and 
coding by using a fixed quantization value by the encoder 
18 (below, the same in Fig. 10 and Fig. 11) , and Fig. 7 
5 and Fig. 8 are graphs of the relationship between the ME 
residual and the real difficulty data D j obtained where 
standard images ["cheer" (cheer leaders), "mobile" 
(mobile and calendar), "tennis" (table tennis), and "diva" 
(diva with noise)] standardized by the CCIR and other 

10 images ("resort") are actually compressed and coded by 

the MPEG system. In Fig. 7 and Fig. 8, an ordinate of the 
graph (difficulty) indicates the real difficulty data D j# 
and abscissa (me resid) indicates the ME residual. 

As understood when referring to Fig. 7 and Fig. 8, 

15 the ME residual has a strong correlation with the real 
difficulty data D^. Accordingly, in place of the real 
difficulty data D j of a picture which becomes a P picture 
or a B picture after compression, the ME residual can be 
used for the generation of the target amount of data T 3 . 

20 R elationship between Flatness and Real Difficulty 

data D J 

Figure 9 is a view of the method of calculation of 
the f latnes s . 

The flatness is calculated by dividing each of the 
25 DCT blocks which serve as the units of DCT processing in 
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the MPEG system into small blocks eaoli consisting of 2 
pixels x 2 pixels as shown in Fig. 9, calculating tlie 
difference of the data of the diagonal pixels (pixel 
values) in these small blocks, comparing the difference 
with a predetermined threshold value, and further finding 
the sum of small blocks in which the difference becomes 
smaller than the threshold value for every picture. 

Note that the value of the flatness becomes smaller 
as the pattern of the video becomes spatially more 
complex and becomes larger when it is flatter. 

Figure 10 is a view of the correlation between the 
flatness and the real difficulty data when generating 
an I picture by the video data compression apparatuses 1 
and 2. 

Note that. Fig. 10 is a graph of the relationship 
between the flatness and the real difficulty data 
obtained when the standard images standardized by the CC1R 
and other images are actually compressed and coded by the 
MPEG system. In Fig. 10, the ordinate of the graph 
(difficulty) indicates the real difficulty data D^, and the 
abscissa (flatness) indicates the flatness. 

As shown in Fig. 10, there is a strong negative 
correlation between the flatness and the real difficulty 
data D J# and it is seen that the real difficulty data 
can be approximated by the method of substituting the 
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flatness into the primary function etc. 

Relationship Between Intra AC and Real Difficulty 

Data 

The intra AC is calculated as the sum of the 
absolute values of the difference between the pixel value 
of each pixel in a DCT block (8x8 pixels) and the average 
value (2) of the pixel values (Pk) in the DCT block for 
every DCT block • That is, the intra AC can be found by 
the following equation 4. 

64 

Mra-AC r £ £ \P-F\ 

picture k=\ 

(4) 

- 1 64 
64 *=i 

Figure 11 is a view of the correlation between the 
intra AC and real difficulty data D^ when generating an I 
picture by the video data compression apparatuses 1 and 
2. 

Note that, Fig, 11 is a graph showing the 
relationship between the intra AC and the real difficulty 
data Dj obtained where the standard images standardized 
by the CCIR and other images are actually compressed and 
coded by the MPEG system. In Fig, 11, the ordinate of the 
graph (difficulty) indicates the real difficulty data D JF 
and the abscissa (intra AC) indicates the intra AC. 

As explained heretofore, it is seen that the real 




difficulty data can be approximated by the primary 
function etc* by each indicator data (statistical amount). 
Accordingly, it is possible to calculate the real 
difficulty data of each picture type as follows. 

The real difficulty data D, is approximated by the 
ME residual (ME,) by the following Equation 5 for a P 
picture and by the following Equation 6 for a B picture. 
Further, the real difficulty data D J is approximated by 
the flatness (FR, ) and intra AC (Intra AC,) or either of 
them by approximation equations (Equations 7 and 8) similar 
to Equations 5 and 6 for an I picture. 

D, = a p x ME, + b p (5) 
D, = a B x ME, + b B (6) 
= a x x FR, + b x ' (7) 

D, = a f j x (intra AC,) + b' x (8) 
Here, a p# a B , a I# and a^. 1 are proportional 
coefficients . 

Further, in the simplified two pass encoding system 
shown in the first embodiment, the target amount of data 
T, was calculated by substituting the real difficulty 
data D, obtained by these approximations into Equation 1 . 

Below, the operation of the video data compression 
apparatus 2 will be explained by taking as an example a 
case where the real difficulty data D, is approximated by 
the ME residual, the flatness, and the intra AC and the 
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noncompressed video data is compressed and coded by the 

simplified two pass encoding system* 

In the encoder control unit 22 , the image 

rearranging circuit 220 rearranges the noncompressed 
5 video data VIN in pictures in the order of coding. The 

scanning transformation and raacroblock forming circuit 

222 performs the picture field transformation etc* The 

statistical amount calculation circuit 224 performs the 

computation processing shown in Fig* 9 and Equation 4 
10 with respect to a picture to be compressed and coded to 

an I picture and calculates a statistical amount such as 

the flatness and intra AC. 

The motion detector 14 generates motion vectors for 

the pictures to be compressed and coded to a P picture 
15 and B picture and further calculates the ME residual* 

The FIFO memory 160 delays the input video data by 

exactly an amount of L pictures* 

The host computer 20 performs the computation 

processing shown in Equation 7 and Equation 3 with respect 
20 to the ME residual generated by the motion detector 14 to 

approximate the real difficulty data and performs the 

computation processing similar to Equation 7 and Equation 

8 to approximate the real difficulty data D j by the 

flatness and the intra AC. 
25 Further, the host computer 20 substitutes the 
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approximated real difficulty data into Equation 1 to 
calculate the target amount of data and sets the 
calculated target amount of data in the quantization 
control circuit 180 of the encoder 18. 

The DCT circuit 166 of the encoder 18 performs DCT 
processing on the j-th picture of the delayed video data. 

The quantization circuit 168 quantizes the data of 
the frequency domain of the j-th picture input from the 
DCT circuit 166 by the quantization value adjusted by 
the quantization control circuit 180 based on the target 
amount of data T.j. 

The variable length coding circuit 170 performs 
variable length coding on the quantized data of the j-th 
picture input from the quantization circuit 168, generates 
the compressed video data VOUT of an amount of data near 
the target amount of data T j# and outputs the same via 
the buffer memory 182 to an external unit. 

Note that, in the TM5 system or the like known as 
the compression algorithm of the MPEG, to calculate the 
quantization value of the macroblock (MQUANT) , a 
statistical amount such as the "activity" shown in the 
following Equation 9 is used. The activity has a strong 
correlation with the real difficulty data D d similar to the 
flatness and the intra AC, therefore it is also possible 
to constitute the video data compression apparatus 2 so 
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as to perform the compression and coding by approximating 
the real difficulty data using the activity in place 
of these parameters. 

act=\+mm Mk -l t Jyar sblk) 

« _ (9) 
var sblk=Y< iff?? 



Where, act^t j-th activity 
sblk: sub block 
var: dispersion. 

Further, while the operation of the video data 
compression apparatus 2 was explained by* taking as an 
example a case where the simplified two pass encoding 
shown in the first embodiment was carried out, needless 
to say the video data compression apparatus 2 can perform 
the predictive simplified two pass encoding as well. 

Further, modifications similar to those with respect 
to the video data compression apparatus 1 shown in the 
first embodiment are also possible with respect to the 
video data compression apparatus 2 shown in the second 
embodiment . 

Third Embodiment 

Below, a third embodiment of the present Invention 
will be explained. 

In the FFRC system shown in the second embodiment, 
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the real difficulty data was approximated by 
substituting the statistically found indicator data 
(statistical amount), that is, the ME residual, flatness, 
intra AC, and activity into the primary function such as 
5 Equation 5 and Equation 6. 

These indicator data and the real difficulty data 
have a strong correlation as shown in Fig. 7, Fig, 8, 
Fig. 10, and Fig. 11, but a slight difference occurs from 
the above primary function according to the pattern of 

10 the video data. 

The processing of the video data compression 
apparatus 2 in the third embodiment is designed so as to 
solve such a disadvantage and is enhanced so that the 
proportional coefficients a^, e^, a I# a r ', etc. shown in 

15 Equation 5 and Equation 6 etc. are adaptively adjusted 
every moment in accordance with the pattern etc. of the 
video data so that the real difficulty data can be 
approximated by the indicator data with a higher 
precision than that in the second embodiment and 

20 compressed video data having a higher quality can be 
produced. 

Below, the processing of the video data compression 
apparatus 2 in the third embodiment will be briefly 
explained. 

25 Whenever the encoder 18 of the video data 
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compression apparatus 2 (Fig. 6) ends one picture's worth 
of compression and coding, tlie host computer 20 learns one 
picture's worth of the amount of data of the generated 
compressed video data and the average value of the 
quantization values at the time of the compression and 
coding and the global complexity explained below can be 
calculated. 

The global complexity is defined as shown in the 
following Equation 10-1 to Equation 10-3 as a value 
obtained by multiplying the amount of data of the 
compressed video data and the quantization value in 
TM5 of the MPEG and indicates the complexity of the 
pattern of the image. 

X z = S x x Q T (10-1) 

X p = S p x Q p (10-2) 
« S B x Q B (10-3) 

Note that, in Equation 10-1 to Equation 10-3, S z . S B , 
and S p indicate the generated amounts of data of the 
coded I picture, B picture, and P picture, Q T . Q B , and Q p 
indicate the average values of the quantization values Q J 
when producing the I picture, B picture, and P picture, 
and X I# X b , and Xp indicate the global complexity of the I 
picture, B picture, and P picture, respectively. 

The global complexities shown in Equation 10-1 to 
Equation 10-3 do not always coincide with the real 
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difficulty data D^, but substantially coincide with the 

real difficulty data Dj so far as the average value of 

the quantization values is not extremely large or small* 

Here, when the indicator data of the I picture, P 
picture, and the B picture, for example the intra AC 
(other parameter is also possible) and ME residual, and the 
global complexity have a proportional relationship, the 
proportional coefficients E x , £ P , and £ B between these 
indicator data and global complexity can be calculated by 
following Equation 11-1 to Equation 11-3. 

Case of I picture (11-1) 

J Intra_AC 



Case of P picture (11-2) 



P X P 



ME_resid 



Case of B picture (11-3) 



ME_resid 



The real difficulty data of each picture type is 
calculated as shown in following Equation 12-1 to 
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Equation 12-3 by using the proportional coefficients e 1 , 
£ p , and £ B calculated by Equation 11-1 to Equation 11-3. 

Case of I picture (12-1) 

D/=e)c Intra _AC j 

Case of P picture (12-2) 

Dj P =ZjX ME_residj 



Case of B picture (12-3) 



The host computer 20 calculates and optimizes the 
proportional coefficients E x , £ P , and £ B whenever the 
encoder 18 compresses and codes one picture as shown in 
Equation 11-1 to Equation 11-3 and finds the values of 
the real difficulty data D 5 of picture types by Equation 
12-1 to Equation 12-3, whereby the real difficulty data 
can be always optimally approximated by the indicator 
data irrespective of the pattern of the video data. 

The host computer 20 performs the computation 
processing shown in Equation 1 with respect to the real 
difficulty data D j approximated as shown in Equation 11 
and Equation 12 to calculate the target amount of data T.. 
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Note that, as In TM5 of the MPEG, when the value of 
the target amount of data T 4 to be actually calculated is 
intentionally changed by a constant ratio with respect to 
the value determined based on the real difficulty data 
D J# the target amount of data can be calculated by the 
following Equation 13-1 to Equation 13-3: 

Case of I picture (13-1) 

r r xKj 

KsKjcD.+Kj, £ D r *Ks £ D. 

P_jjicturt B_picti<re 



Case of P picture (13-2) 

10 

r ; S3 xRj 

KfKfD.+K/c £ W £ D, 

P ^picture B _pictitre 



Case of B picture (13-3) 



T 53 xRj 

KpcKjcD^Kjc £ D f +Kp £ D, 

P_picture B ^picture 



15 



Note that, in all denominators of Equation 13-1 to 
Equation 13-3, D T p B indicates the real difficulty data 
approximated by the indicator data produced from L 
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pictures' worth of the noncompressed video data buffered 
in tlie FIFO memory 160 before input to the encoder 18, 
and Rj indicates the average value of the amounts of data 
which can be assigned to L pictures after the j-th picture* 
5 Below, the operation of the video data compression 

apparatus 2 according to the third embodiment will be 
explained. 

Figure 12 is a view of the compression and coding 
operation of the video data compression apparatus 2 (Fig. 
10 6) in the third embodiment. 

The encoder control unit 22 rearranges the pictures 
of the noncompressed video data VIN in the order of 
coding similar to the second embodiment, performs the 
picture field transformation, etc. and calculates the 
15 statistical amount such as flatness and intra AC from the 

j+N-th picture to be compressed and coded to the I picture. 

The motion detector 14 produces a motion vector for 
the j+N-th picture to be compressed and coded to the I 
picture or B picture similar to that in the first 
20 embodiment and second embodiment and further calculates 
the ME residual (Fig. 12A) . 

The FIFO memory 160 delays the input video data by 
exactly the amount of I* pictures similar to the first 
embodiment and the second embodiment. 
25 The host computer 20 performs the computation 
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processing shown ±n Equation 12-1 and 12-2 with respect to 
the ME residual produced by the motion detector 14 , 
approximates the real difficulty data D^, performs the 
computation processing shown in Equation 12-3, and 
approximates the real difficulty data D d by the intra AC, 
etc. (Fig, 12B). 

Further, the host computer 20 substitutes the 
approximated real difficulty data D J into Equation 1 or 
Equations 13-1 to 13-3 to calculate the target amount of 
data T 3 and sets the same in the quantization control 
circuit 180 of the encoder 18 {Fig. 12C) . 

The DCT circuit 166 of the encoder 18 performs the 
DCT processing for the j-th picture of the delayed video 
data similar to that in the first embodiment and the 
second embodiment. 

The quantization circuit 168 quantizes the data of 
the frequency domain of the j-th picture input from the 
DCT circuit 166 by the quantization value adjusted 
based on the target amount of data T^ by the quantization 
control circuit 180 and, at the same time, calculates the 
average value of the quantization values used for the 
compression and coding of the j-th picture and outputs 
the same to the host computer 20. 

The variable length coding circuit 170 performs 
variable length coding on the quantized data of the j-th 
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picture input from the quantization circuit 168 similar to 
that in the first embodiment and second embodiment to 
produce the compressed video data VOUT of an amount of 
data near the target amount of data and outputs the 
same via the buffer memory 182. 

When the encoder 18 terminates the compression and 
coding of the j-th picture, the host computer 20 
calculates the global complexity as shown in Equation 10- 
1 to Equation 10-3 based on the average value of the 
quantization values with respect to the j-th picture 
input from the quantization control circuit 180 and the 
amount of data of the compressed and coded j-th picture 
(Fig, 12D). 

Further, the host computer 20 updates the 
proportional coefficients e 1 , £ P # and £ B as shown in 
Equation 11-1 to Equation 11-3 by the calculated global 
complexity (Fig. 12E). The updated proportional 
coefficients E 1 , £ P , and £ B are reflected in the 
transformation equations at the time of compression and 
coding of the next picture (Equation 12-1 to Equation 12- 
3). 

The content of the processing of the host computer 
20 in the third embodiment will be further explained by 
referring to Fig. 13. 

Figure 13 is a view of the content of the processing 
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of the host computer 20 (Fig* 6) of the video data 
compression apparatus 2 in the third embodiment. 

As shown in Fig* 13, at step 300 (S300), the host 
computer 20 fetches the (j+L)-th indicator data 
(statistical amount) such as ME residual or intra AC from 
the encoder control unit 22 or the motion detector 14. 

At step 302 (S302), the host computer 20 decides to 
which picture type the (j+l)-th picture is to be 
compressed and coded. When the (j+l)-th picture is to be 
compressed and coded to an I picture, the operation 
routine proceeds to the processing of S304; when it is to 
be compressed and coded to a P picture, the operation 
routine proceeds to the processing of S306; and when it 
is to be compressed and coded to a B picture, the 
operation routine proceeds to the processing of S308. 

In each of step 304 (S304), step 306 (S306), and 
step 308 (S308), the host computer 20 approximates the 
real difficulty data by Equation 12-1 to Equation 12-3. 

At step 310 (S310), the host computer 20 calculates 
the target amount of data by Equation 1 or Equation 
13-1 to Equation 13-3 by using the approximated real 
difficulty data D 3 . 

At step 312 (S312), the encoder 18 compresses and 
codes the j-th picture. 

At step 314 (S314), the host computer 20 calculates 
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the global complexities X i# X^, and X p [X (I, B, P)] from 
the amount of data of the j-th picture compressed by the 
encoder 18 and the average value of the quantization 
values Q d set in the quantization circuit 168 by the 
quantization control circuit 180 . 

At step 316 (S316), the host computer 20 decides to 
which picture type the (j+l)-th picture is to be 
compressed and coded. Where the (j+l)-th picture is to be 
compressed and coded to an I picture, the operation 
routine proceeds to the processing of S318; when it is to 
be compressed and coded to a P picture, the operation 
routine proceeds to the processing of S320; and when it 
is to be compressed and coded to a B picture, the 
operation routine proceeds to the processing of S322. 

In each of step 318 (S318), step 320 (S320)m and 
step 322 (S322), the host computer 20 updates the 
proportional coefficients e 1 , £ P , and £ B by Equation 11-1 to 
Equation 11-3. 

At step 324 (S324), the host computer 20 increments 
the numerical value j . 

Note that, as mentioned in the second embodiment, 
there is a case where an offset (5 P ) exists between the 
real difficulty data and the multiplied value of the 
proportional coefficients C 1 , £ P , and £ B -and the indicator 
data. In such a case, by dividing the values obtained by 
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subtracting the offset values 5 1 , 5 P , and 5 B from the 
global complexities X x , X,,, and X p by the indicator data, 
the proportional coefficients E 1 , e P , and £ B can be 
calculated. 

Further, modifications similar to those shown in the 
second embodiment are also possible for the operation of 
the video data compression apparatus 2 shown in the third 
embodiment . 

As mentioned above, according to the operation of 
the video data compression apparatus 2 in the third 
embodiment, the same effect as that by the operation of 
the video data compression apparatus 2 shown in the second 
embodiment is obtained and, in addition, a further 
correct target amount of data T d than that in the second 
embodiment can be calculated, and as a result, the 
quality of the compressed video data can be improved. 

Fourth Embodiment 

Below, a fourth embodiment of the present invention 
will be explained. 

Up to here, the explanation was made of the feed 
forward rate control (FFRC) system for achieving both an 
improvement of quality of the compressed video data and a 
real time property of the compression and coding 
processing by using the indicator data (statistical 
amount), that is, flatness, intra AC, activity, and ME 
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residual, as the second embodiment and the third 
embodiment . 

In the fourth embodiment, an explanation will be 
made of a video data compression method (improved FFRC 
5 method) using a video data compression apparatus 2 (Fig, 
16 and Fig. 17) with which the encoder for obtaining the 
real difficulty data is unnecessary and in addition 
the quality of the compressed video data of the border 
(scene change) part of the video data (scene) contained 
10 in the edited video data is not lowered. 

P It can be considered to perform the compression and 

m 

f!f coding by detecting a part where the real difficulty data 

O D d largely changes along with time as a scene change part 

I** 

and changing the picture type sequence. The detection of 
15 such a scene change is possible also in the FFRC method 
by monitoring the change of the real difficulty data D d 
approximated by the indicator data in place of the real 
difficulty data along with the elapse of time. 

However, to determine the existence of a scene 
20 change, it is necessary to monitor the change of the 

indicator data within a range of about 1 "GOP in time before 
and after the scene change part. In the video data 
compression apparatus 2, the detection of the scene 
change part becomes possible only after a considerable 
25 time has elapsed after the motion detector 14 calculates 
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the Indicator data* In actuality, there also exists a 
possibility that the detection of the scene change part 
will become possible only immediately before the 
compression and coding processing in the encoder 18. 
5 Accordingly, the host computer 20 must substantially 

terminate the processing for approximating the real 
difficulty data by the indicator data (Equations 5 to 8 
shown in the second embodiment and Equation 12-1 to 
Equation 12-3 shown in the third embodiment) before the 

10 detection of the scene change in order to ensure 
sufficient processing time* 

The video data compression apparatus 2 in the fourth 
embodiment provisionally performs the processing for 
approximation of the real difficulty data by the 

15 indicator data or the global complexity in a state where 
the result of detection of the scene change is not 
determined, corrects only the part requiring a change along 
with a scene change in the provisionally calculated real 
difficulty data after the existence of a scene change 

20 and the existence of a change of the picture type sequence 
are determined, and thereby performs the processing for 
calculating the target amount of data . 

Below, the content of the compression and coding 
processing of the video data compression apparatus 2 in 

25 the fourth embodiment will be explained by taking as an 




example a case where the picture type sequence with 
respect to N number of pictures Is finally determined 
whenever the calculation of the ME residuals of N [for 
simplification of the explanation, below, for example it 
is assumed that N = L (L is the number of pictures 
corresponding to the delay time of the FIFO memory 160)] 
number of pictures is carried out* Note that, the N 
number of pictures used for the determination of the 
picture type sequence are the processing units of the 
processing for determining the picture type sequence and 
do not always have to coincide with the picture type 
sequence in the encoder 18. Further, the header does not 
have to be an I picture unlike the usual GOP. Further, 
below, one set of such N number of pictures will be also 
referred to as a rate control GOP (RCGOP). 

Figure 14 is a view of the compression and coding 
operation of the video data compression apparatus 2 in 
the fourth embodiment (Fig. 5) in the order of coding. 

The motion detector 14 generates a motion vector for 
the (j+N)-th picture to be compressed and coded to a P 
picture and a B picture similar to that in the first 
embodiment to the third embodiment and further calculates 
the ME residual. 

The encoder control unit 22 performs the 
preprocessing of the rearrangement etc. of pictures 
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similar to that in tlie second, embodiment and the third 
embodiment and further calculates the indicator data such 
as the flatness, intra AC, and activity. 

The FIFO memory 160 delays the input video data by 
5 exactly an amount of L pictures similar to that in the 
first embodiment to the third embodiment. 

Whenever the video data compression apparatus 2 
(Fig. 5) ends one picture's worth of compression and 

III 

coding, the host computer 20, similar to that in the second 

U 

Ifl 10 embodiment and the third embodiment, receives as input 

if, 

Kl the flatness, intra AC, and activity calculated by the 

111 encoder control unit 22 and the MR residual (statistical 

tf 

p amount) calculated by the motion detector 14. The host 

computer 20 stores these indicator data (Fig. 14A) . 

15 Further, assuming that no scene change occurs and no 
change will occur in the picture sequence, the host 
computer 20 approximates and predicts the real 
difficulty data in the case assuming that no scene 
change exists by Equation 12-1 to Equation 12-3 by using 

20 the optimized proportional coefficients 8 1 , and e B 

(Equation 11-1 to Equation 11-3 shown in the third 
embodiment) similar to that in the third embodiment (Fig. 
14B). 

Specifically, assuming that the N-th picture from 
25 the I picture of the first RCGOP is compressed and coded 




to an I picture , the pictures of the order of whole 
multiples of M (n x M) are compressed and coded to P 
pictures, and the pictures other than them are compressed 
and coded to B pictures, the host computer 20 substitutes 
5 the indicator data produced by the pictures to be 

respectively compressed and coded to an I picture, P 
picture, and B picture and the proportional coefficients 
£*, £ P , and £ B into Equation 12-1 to Equation 12-3 to 
approximate and calculate the real difficulty data • 

10 Note, M indicates the interval of the P pictures in the 
encoder 18 where there is no scene change. 

That is, the host computer 20 counts the number of 
pictures by using the I picture of the previous RCGOP 
(first CGOP: RCG0P # 1) as a reference, assumes which 

15 picture type the encoder 18 compresses and codes each 
picture of the second RCGOP (RCGOP f 2) to, and 
approximates and predicts the value of the real difficulty 
data by the indicator data in accordance with the 
assumed picture type as shown in Equation 12-1 to 

20 Equation 12-3. 

Note that, the probability of existence of the scene 
change part in the RCGOP is considered to be relatively 
small, therefore the host computer 20 will calculate the 
target amount of data with respect to most RCGOP based 

25 on the predicted real difficulty data (Fig. 14F). 
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Further, the real difficulty data D 3 is only used in 
the calculation of denominator of Equation 1 (first 
embodiment) or Equation 13-1 to Equation 13-3 (third 
embodiment). As will be explained later, the host 
computer 20 performs correction at a stage where the 
existence of a change of the picture type sequence is 
determined, therefore the value of the target amount of 
data can be always correctly calculated. 

When the calculation of the real difficulty data 
of each picture of the second RCGOP (RCGOP # 2) is 
terminated, the host computer 20 can detect a scene 
change in the second RCGOP by using the calculated real 
difficulty data or indicator data. For example, the 
host computer 20 monitors the change of the value of the 
real difficulty data D d along with the elapse of time and 
determines that a scene change is generated in a P 
picture by the method of detecting the P picture in which 
the value of the real difficulty data indicates a 
value of for example 1.5 (practically, preferably a 
value within a range of from 1.4 to 1.8) times the real 
difficulty data D j x of the P picture immediately before 
this, and so on. Where the scene change is detected, the 
host computer 20 further controls the encoder 18 and 
changes the picture type sequence when compressing and 
coding the parts before and after the scene change of the 
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edited video data so as to change the first P picture of 
the back scene to an I picture not referring to the last 
picture of the previous scene and change the last I 
picture of the previous scene to the P picture as shown 
5 in Figs. 15B and 15C. 

By such processing of the host computer 20, the 
existence of a change of the picture type sequence is 
seen, and the picture type to which each picture is to be 
compressed and coded is decided (Fig. 14C) . 

10 The host computer 20 corrects the value of the real 

difficulty data for the second RCGOP based on the 
stored indicator data and the picture type after the 
change where there is a change in the picture type 
sequence, calculates the correct real difficulty data 

15 (Fig. 14D), calculates the target amount of data T N+1 

(target bit) of the (N+l)-th picture in accordance with 
each picture type by using Equation 1 and Equation 13-1 
to Equation 13-3 (Fig, 14E), and sets the same in the 
quantization control circuit 180 of the encoder 18. 

20 Specifically, as shown in Fig. 15C, the host 

computer 20 substitutes the indicator data of the picture 
changed so as to become not the P picture, but the I 
picture after the compression into Equation 12-2 in place 
of Equation 12- 1. and conversely, substitutes the 

25 indicator data of the picture changed so as to become not 
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the I picture, but the P picture after the compression 
into Equation 12-1 in place of Equation 12-2, thereby to 
correct the value of the real difficulty data D^. 

The DCT circuit 166 of the encoder 18 performs DCT 
5 processing similar to that in the first embodiment to the 
third embodiment* 

The quantization circuit 168 quantizes the DCT- 
processed video data by the quantization value 
111 adjusted by the quantization control circuit 180 based on 

10 the target amount of data T d and calculates the average 

O value of the quantization values Q . 

iy 

f IJ The variable length coding circuit 170 performs 

P 

P variable length coding similar to that in the first 



r 



embodiment to the third embodiment, produces the compressed 
15 video data VOUT of an amount of data near the target 

amount of data T^, and outputs the same via the buffer 
memory 182. 

When the encoder 18 terminates the compression and 
coding of the j-th picture, the host computer 20 
20 calculates the global complexity as shown in Equation 10- 
1 to Equation 10-3 based on the average value of the 
quantization values Q j# and the amount of data of the 
compressed and coded j-th picture* 

Further, the host computer 20 updates and optimizes 
25 the proportional coefficients e 1 , E v , and £ B as shown in 




Equation 11-1 to Equation 11-3 by the calculated global 
complexity. In the same way as in the third embodiment , the 
updated proportional coefficients E 1 , £ P , and € B are 
reflected in the transformation equations at the time of 
compression and coding of the next picture (Equation 12-1 
to Equation 12-3). 

The content of the processing of the host computer 
20 in the fourth embodiment will be further explained 
referring to Fig. 16. 

Figure 16 is a flowchart of the content of the 
processing of the host computer 20 in the video data 
compression apparatus 2 in the fourth embodiment. 

As shown in Fig. 16, the processing of the host 
computer 20 in the fourth embodiment is divided into a 
first stage (S400) and a second stage (S420). At the 
first stage, processing for predicting the real 
difficulty data is carried out by assuming that there 
is no scene change and there is no change in the picture 
type sequence, and at the second stage, processing for 
correcting the value of the real difficulty data is 
carried out where a scene change occurs and the picture 
type sequence is changed. 

The first stage (S400; S402 to S412) is the 
processing for predicting the real difficulty data 
where there is no scene change. At step 402 (S402) of the 
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first stage, the host computer 20 fetches the indicator 
data (statistical amount) such as the ME residual or 
intra AC of the (j+L)-th picture from th€5 encoder control 
unit 22 or the motion detector 14 and stores the same. 

At step 404 (S404), the host computer 20 decides 
whether or not the [j+L (j+N)]-th picture is to be 
compressed and coded to a B picture. Where the (j+I*)-th 
picture is to be compressed and coded to a B picture, the 
operation routine proceeds to the processing of S406, 
while where it is not to be compressed and coded to a B 
picture, the operation routine proceeds to the processing 
of S408. 

At step 406 (S406), the host computer 20 predicts 
that the (3+L)-th picture is to be compressed and coded 
to a B picture and approximates and calculates the real 
difficulty data by Equation 12-3. 

At step 408 (S408), the host computer 20 decides 
whether or not the number (interval) from the picture to 
be compressed and coded to the I picture in the previous 
RCGOP to the (j+L)-th picture of the current RCGOP is N. 
Where the interval is N, the operation routine proceeds 
to the processing of S412, while where it is not N, the 
operation routine proceeds to the processing of S410. 

At step 410 (S410), the host computer 20 predicts 
that the (j+L)-th picture is to be compressed and coded 




to a P picture and approximates and calculates tlie real 
difficulty data by Equation 12-2. 

At step 412 (S412), the host computer 20 predicts 
that the (j+L)~th picture is to be compressed and coded 
5 to an I picture and approximates and calculates the real 
difficulty data by Equation 12-1 . 

The second stage (S420; S422 to S434) is the 
processing for correcting the real difficulty data D d 
jjji predicted at the first stage. At step 422 (S422) of the 

10 second stage, the host computer 20 decides whether or not 
O a new RCGOP starts* Where it does not start, the 

3 - £ 

operation routine proceeds to the processing of S430, 
P while where it starts, the operation routine proceed to 

the processing of S424. 

15 At step 424 (S424), the host computer 20 decides 

whether or not the picture type sequence is changed so 
that the position of the I picture changes. Where the 
picture type sequence is changed so that the position of 
the I picture changes, the operation routine proceeds to 

20 the processing of S426, while where it is not changed, 

the operation routine proceeds to the processing of S430. 

At step 426 (S426), the host computer 20 
approximates and calculates the real difficulty data 
by Equation 12-1 for the picture newly compressed and 

25 coded to the I picture. 
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At step 428 (S428), the host computer 20 
approximates and calculates the real difficulty data 
by Equation 12-2 for the picture newly compressed and 
coded to the P picture. 
5 At step 430 (S430), the host computer 20 calculates 

the target amount of data with respect to the j-th 
picture by Equation 1 or* Equation 13-1 to Equation 13-3 
; and sets the same in the quantization control circuit 180 

|f| of the encoder 18 (Fig. 5). 

10 At step 432 (S432), the encoder 18 compresses and 

Q codes the j-th picture based on the target amount of data 

fU T set in the quantization control circuit 180 . 

pi 

D At step 434 (S434), the host computer 20 increments 

the numerical value j . 

15 Note that, in the fourth embodiment, the host 

computer 20 of the video data compression apparatus 2 
performs the processing for correcting only the real 
difficulty data D j of pictures changed after compression 
where there is a scene change, but when there is a room 

20 in the processing time, it can be modified so that the real 
difficulty data of all pictures are calculated after 
the picture type sequence is decided. 

Further, modifications similar to those in the first 
embodiment to the third embodiment are also possible for 

25 the operation of the video data compression apparatus 2 
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shown ±n the fourth embodiment. 

Further , the contents of the processing of the video 
data compression apparatuses 1 and 2 respectively 
explained in the first embodiment to the third embodiment 
can be combined so far as they do not contradict each 
other . 

As mentioned above, according to the operation of 
the video data compression apparatus 2 in the fourth 
embodiment, the same effect as that by the operation of 
the video data compression apparatus 2 shown in the second 
embodiment and the third embodiment is obtained, a 
further correct target amount of data than that in 
those embodiments can be calculated, and in addition the 
quality of the compressed video data of the scene change 
part is not lowered. 

As explained above, by the video data compression 
apparatus according to the present invention and the 
method of the same, audio and/or video data can be 
compressed and coded to a predetermined amount of data or 
less by a method other than the two pass encoding. 

Further, by the video data compression apparatus 
according to the present invention and the method of the 
same, the video data can be compressed and coded in 
almost real time and in addition a high quality image can 
be obtained after expansion and decoding. 
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Furthers by the video data compression apparatus 
according to the present invention and the method of the 
same, the compression rate is adjusted by estimating the 
amount of data after the compression and coding and the 
compression and coding processing can be carried out by a 
method other than two pass encoding. 



